CDROM.txt 6.04 HOW TO BUILD APRS MAPS FROM THE USGS CD ROM 19 Nov 94 As of version 6.04, all information on using the original 2,000,000:1 USGS CD ROM has been moved to the end of this file, since by now, (fall 94) almost all of the US population centers have been made into APRS maps at this level of detail. This FILE now begins to discuss the use of 100,000:1 USGS CD ROMS that contain data on roads all the way down to your street. Because of the tremendous amount of data available in this data base, USGS sells the data on a set of 13 CD's. To see which CD covers your area, hit 1 on your APRS screen to get to the whole USA map. Then press MAPS-ALT command to see the CDROM.map file which is under the USA map. This display shows you the areas of the 13 different ROMS. My yellow lines are crude, but the data in the files does truncate exactly at the state lines. To obtain one of these CD ROMS, send $32 for each ROM to the USGS Earth Science Info Center, 507 National Center, Reston, VA 22092. 703-648-6045 or call 1-800-USA-MAPS. BE SURE TO SPECIFY THE OPTIONAL FORMAT!!!!!!!!!!!!!!!! (vice standard) WARNING: Since both Ken, W7KKE and myself are continuing to improve these programs, and keep modifying each others code, the following programs are identical, except for the latest mods installed by each writer. Look at the latest file date, to see which one is the most recent: Ken's: CONV100.bas = Bob's 100toGRF.bas DEFRAG.bas = JOIN.bas JUST BECAUSE WE HAVE NOW WRITTEN ROUTINES TO HELP YOU MAKE MAPS FROM BOTH THE 2,000,000 and 100,000 SCALE USGS CD ROMs, DO NOT GIVE UP ON MAKING YOUR OWN MAPS USING ANY OF THE OTHER METHODS. THE 2,000,000 SCALE CD IS ONLY GOOD FOR 64 MILE MAPS because THEY DONT HAVE ENOUGH DETAIL TO MAKE SMALLER MAPS, and they HAVE TOO MANY POINTS TO MAKE MAPS MUCH LARGER! The 100,000 scale maps generate so many points that an APRS map of city streets can be no larger than about 1 mile in dense neighborhoods! BACKGROUND: There are 3 problems with the USGS data that causes excessive points in an APRS map. First, lines begin and end at EACH intersection. For am interstate, for example, you end up with dozens of short road segments wasting 3 point pairs per exit. Second, these short segments are not always consecutive in the file, making it extremely time consuming to try to join them. Third, the USGS data guarantees a point every .01 inch at the original maps scale (1/3 mile for the 64 mile maps) whether it is needed or not. A 10 mile long straight road is stored as 30 points in USGS vice 2 points in APRS. At the 100,000:1 scale, .01 inch is only 100 feet! VERY FINE DETAIL USING THE USGS 100,000:1 DETAIL CD ROMS W7KKE wrote a program called CONV100 that I have modified to 100toGRF.bas which can take data from the 100,000 scale CD ROMS. It generates so many roads in an urban environment, that just a 1 square mile map has over 5000 points. I have upped the APRS maximum limits to accomodate up to 3600 points but you should avoid making these large size maps unless you absolutely need every street, because the large size also means APRS takes longer to load maps! Since we are vehicle tracking for radio purposes, a whole map 1 mile square is only one one-thousandth of the potential coverage area of one APRS digipeater, so you can see that the effort to make these maps is not worth the time in my opinion. (unless you are making the detail for a commercial application and are willing to spend the time!) Here is how to do it: 1. Run the USGS EXTRACT program and follow its instructions to output the appropriate map data for the 15 x 15 minute area that you are interested in. These files will be of the form XXXRDSnn for ROADS and xxxHYFnn for hydro where the nn is for multiple files to cover the area. In my urban area, each of the ROAD files covers about a 2 mile area. 2. Run 100toGRF.bas and once you give it one of the above file names, it will automatically generate a series of 10 output files of the form PREnRD.GRF for n different ROAD files and two other files preWB.grf and preST.grf for waterbodies and streams. 100toGRF.bas outputs these files in the same format as the original 2,000,000 that MAPFIX knows how to read. The PRE is a 3 or 4 letter prefix that you specify for this series to keep it separate from other efforts. So far, the number 4, 9 and 10 maps in my area have been empty... but the pre8RD.GRF file is the largest by a factor of ten! It contains all of the neighborhood streets. 3. Run MAPFIX and use the B command to establish the boundary of the area that you want to develop. Choose a scale of 2400, 3000, 3600, 4800 or so that will cover the total area you eventually want covered. For city streets, you should use at least 2400 or more. Next use the alt-USGS command to bring in the first of the above files. Then do the alt-SMOOTH and the alt-JOIN commands to minimize the map. Then save it with a new name. Then repeat this step for all the maps that were output from the 100toGRF.bas program. After you have saved each of these, you will now have almost a dozen maps each containing a different feature, but in APRS format. WARNING!: NEVER do a second ALT-U USGS BUILD on the same map, because it will clear all current points and start a new map from scratch. 4. Next, run my JOIN.bas program, or Ken's better DEFRAG.bas program on EACH of the files generated by MAPFIX in the above step 3. These programs go through the files over and over again, trying to join together all of the disjointed road segments (of the same class) generated by the USGS data base. Hopefully you may see a 20% or more reduction in points. Re-save each of these reduced maps under a new name so that you do not distroy your originals if there is a problem. NOTE: Be sure to also review the steps at the end of this file on how MAPFIX is used to read the 2,000,000:1 CD ROMS. Since the 2,000,000:1 process was developed first and is the basis for MAPFIX routines, you might find some helpful details or clarifying information. Ken Swaggart, W7KKE, wrote the conversion routines to put the 100,000:1 data into the 2,000,000:1 format. 4. Now, combine these maps into one APRS map, using the MAPFIX alt-IMPORT command. This command will bring just one, or all of the map features of one file into the current file. In an urban area, the level 8 road file will be larger than all the others combined. You can omit it to allow your map to cover a larger area or include it for the finest detail available. Good luck. I am just beginnning to play with this level of detail, and to this point, it is far more practical to use a digitizer! For more detail on the use of MAPFIX with CD ROMs, also read the 2,000,000:1 section below, since it evolved and was the basis for APRS maps before we began messing with the 100,000:1 data. TO SEE KEN's ORIGINAL EXPLAINATION ON HOW TO DO THIS, SEE HIS README.100 FILE. PROCEDURES FOR 2,000,000:1 CD ROM THAT COVERS THE WHOLE US 1. Use USGS XTRACT.EXE program to extract the CD ROM data in the GRAPHIC format for your area from one of the dozen or so major USGS areas. Then select up to 6 files for extraction onto your HARD disk, one for each of the data types, BOUNDARIES, ROADS, WATER BODIES, STREAMS, RAILROADS, and HYPSOGRAPHY (continental divide). These files will take about 5 MBytes. To save HD space, and also to end up with the minimum of points, you will probably only use the ROADS, WATERBODIES, and STREAMS files. Note the filename assigned by XTRACT. 2. Run MAPFIX.BAS program and load the best existing APRS map that covers the desired area. This map is only used to help you locate the area of interest. Next, move the cursor to the center of interest and select a SCALE using the BOUNDARY (B) command. This command will show you the maximum map possible for a given scale in Points/per degree. I have found 600 to be about right. 450 in sparse states and 300 in UTAH and NEW Mex. If you use any less, to get larger maps, you will end up with far too many points. I used 450 for the SouthCarolina map, and it took me 6 hours to get the resulting 3000 points down below 1500! In the East, I used 900 for a 35 mile map around Philidelphia. Making maps any smaller is probably not worth the effort, since there will be so few roads in it.. 3. When you have the B command set to the desired number of Pixels-per-degree, press the alt-U key. MAPFIX will allow you to round the ORIGIN values, to reasonable nearly whole numbers or fractions of a degree so that later manual operations with a map and ruler will have round numbers to deal with. 4. Next APRS asks you for the type of file (RD, ST, WB, etc) and then the exact path to the XTRACTed USGS data. This path should include the first part ot the filename up to the (_) underline that separates the RD or WB etc final two characters. MAPFIX will then scan the entire indicated file for all points within the selected border. While converting the points to APRS format, MAPFIX will also compare the slope of each new point with the previous point and ignore the point if it has approximately the same slope (within 1.2) The total number of points will be from 2000 to 4000. 4. Next MAPFIX has a SMOOTHER command which will run another slope comparison process. You can select the SMOOTHER factor. A factor of 1 will only throw out points that are exactly the same slope (few if any). A factor of 1.2 (best) will throw out points whose slope differs by less than 1.2. Bigger values will throw out more points (usually including some necessary ones). I never use a value greater than 1.5. To test the smoothing algorithm, run the TEST SMOOTHER command which will mark every point that would be eliminated with a red dot, and every point that will be kept with a yellow dot; but without actually eliminating any points. Look at the map closely and see if you can accept the decisions of the SMOOTHER. Select your chosen factor and run SMOOTHER. (alt-S) 5. Next use the alt-JOIN command. This will cause MAPFIX to scan through the map looking for consecutive map features that end and then start at the same point. These two features will be joined into one longer continuous feature. At this point, you can save the map, quit MAPFIX and run MYdefrag.bas which extends this same philosophy by comparing each feature to EVERY other feature (not just consecutive) still trying to match end points of one feature with the start of another. This process can usually eliminate another couple of hundred points. The result is saved in MAPTEMP.map. 6. Now you have the smallest APRS map that can be done automatically. Reload MAPFIX, reload the MAPTEMP.map (or other optional name) and Move the MAP POINTER to the beginning FEATURE using the RESET command and use the gray +/- keys to cycle through EVERY point in the file. Use the alt-D key to delete all unnecessary points. Look for complete duplicates of some lines too. This should get you down to below 2000 or so. 7. Notice that you only have one category of features so far (usually roads). Unless you require county lines I recommend that you ignore them. The USGS BOUNDARIES file contains even more points than the ROADS file! AND the SMOOTHER routine does not work well, because property lines are very jagged, unlike smooth curves in roads. The same thing goes for WATER BODIES and STREAMS. 10. To allow you to add some detail from these other USGS files, MAPFIX has the USGS OVERLAY feature which will overlay the USGS data on the MAPFIX screen without doing anything. Then you can use the NEW-FEATURE and ADD commands to draw APRS features over the top of these temporary points. The philosopy here is that 96% of amateur interest is on roads, and you can sure burn up points on waterways and county lines. For rivers, about the only points I use are where the water goes under a bridge. Most folks have no idea where a river is out in the boonies (and could care less). Of course, if you are making a map for boats with GPS, then start the process with the WATERBODIES file and add just enough roads for them to find the boat ramps.. You will first need to overlay on the 64 mile scale to see where things are, but then zoom into the 8 mile scale at least and do the overlay again to get enough resolution for drawing a particular feature. You can use the NEW, ADD and MOVE commands, but any other commands will redraw the map and wipe-out the overlay. I have added the Ctrl-R command to toggle the map REDRAW process on and off. This may help you to add lines without destroying the USGS overlay. You can do the overlay again and again. WARNING!: NEVER do a second ALT-U USGS BUILD, because it will clear memory and start from scratch. CAUTION: Do not expect too much from this CD ROM. ALthough the 2,000,000:1 USGS database seems to have resolution down to 1/3 rd of a mile, remember that that 1/3rd of a mile was actually 1/100th of an inch on the original map! Your 64 mile section was only a two inch square!!!!! AND THE DATA WAS PROBABLY DERIVED BY A HIGH SCHOOL SUMMER STUDENT using a mouse! I have found this data to be very crude, and although the maps lines look smooth, you are getting about the same quality as if you were looking at a state highway map through a magnifying glass! If you want good detail, use MAPFIX and a digitizer on a much better original map, such as a 100,000:1 or even 24,000:1 (which is the standard USGS 7.5 minute map). There you can get down to your driveway! IT STILL TAKES AN EXCESSIVE AMOUNT OF MANUAL EFFORT TO REDUCE THESE MBYTES OF USGS DATA DOWN TO A reasonably quick loading 15K APRS MAP! IT TAKES ME 4 to 12 HOURS PER MAP! By now (Fall 94) I have done most of the US to the state highway level of detail. IF YOU WANT ME TO DO A SPECIAL MAP OF YOUR AREA, to the resolution in the 2,000,000:1 ROM, SEND $$$ or convince me it is in the general best interest of all HAMS. THAT ALONE SHOULD KEEP YOU INTERESTED IN DOING YOUR OWN. HI HI FOR THE 100,000:1, I will only do these maps commercially, because the 3 days worth of effort only results in a map that covers a few miles. Only a few HAMS will even live inside one of these small areas. If you are a MAP-ology expert, or an ace programmer and can help us figure out a way to better automate the process, then we would love to make the whole process easier for everyone. Contact Ken Swaggart, W7KKE in Calif, or myself.